Playing Games with Ais: The Limits of GPT-3 and Similar Large Language Models

Abstract This article contributes to the debate around abilities of large language models such as GPT-3, dealing with: firstly, evaluating how well GPT does in Turing Test, secondly limits models, especially their tendency generate falsehoods, and thirdly social consequences problems these have with truth-telling. We start by formalising recently proposed notion reversible questions, which Floridi & Chiriatti (2020) propose allow one ‘identify nature source answers’, a probabilistic measure based on Item Response Theory from psychometrics. Following critical assessment methodology led previous scholars dismiss GPT’s abilities, we argue against claims that GPT-3 completely lacks semantic ability. Using ideas compression, priming, distributional semantics webs offer our own theory like can competently engage various tasks. The real reason answers seem senseless being truth-telling is not amongst them. claim kinds cannot be forced into producing only true continuation, but rather maximise objective function they strategize plausible instead truthful. This, moreover claim, hijack intuitive capacity evaluate accuracy its outputs. Finally, show this analysis predicts widespread adoption generators tools for writing could result permanent pollution informational ecosystem massive amounts very often untrue texts.